IEEE 2004 May 151[2]

INTERNET TECHNOLOGY SERIES Performance of New Link State Advertisement Mechanisms in Routing Protocols with Traffic Engineering Extensions Hussein M. Alnuweiri, Lai-Yat Kelvin Wong, and Tariq Al-Khasib, University of British Columbia ABSTRACT The prevalent use of best-effort topologydriven IP routing protocols with shortest path calculations can often lead to serious imbalance of packet traffic distribution when least cost paths converge on the same set of links, leading to unacceptable delays or packet loss even in the presence of feasible paths over less utilized links. Recently proposed enhancements to common routing protocols are promising to overcome such shortcomings by providing the means to distribute link state information that is more pertinent to traffic engineering in routed networks. This article presents several key results on the performance of the recently proposed OSPF-TE, with particular emphasis on OSPFTE protocol traffic overhead and the impact of new link state advertisement triggering mechanisms on traffic-engineered routing accuracy. INTRODUCTION The high cost of network equipment assets and the commercial competitive nature of providing Internet services have been pressing large Internet service providers (ISPs) to look for ways to generate more revenue without substantially increasing their investments in upgrading network infrastructure. A key to achieving such goal is the ISPs’ ability to provide high-value IP services to customers, while lowering their network costs by achieving maximal operational efficiency. Emerging traffic engineering (TE) techniques are promising to enable ISPs to optimize their resource utilization by evenly distributing or balancing the traffic load on the various links while maintaining the promised levels of services, or quality of service (QoS), to customer traffic flows. Currently used routing protocols, however, have little or no support for TE requirements. The main target of most IP routing protocols is to achieve connectivity between net- work routers and eliminate loops. The prevalent use of topology-driven routing protocols with shortest path calculations often leads to serious imbalance of traffic volume when least cost paths converge over the same set of links and router interfaces, leading to traffic congestion with unacceptable packet delays or packet loss. Such service-affecting inefficiencies can occur dynamically (i.e., unpredictably), and despite the presence of feasible paths over less utilized links undiscovered by the shortest path algorithms employed by the network routers. To solve some of the above problems, a number of algorithms and methodologies have been proposed recently, mostly based on the concept of routing with resource reservation using constraint-based routing or QoS-based routing algorithms. Deploying such techniques in the IP network requires either defining new routing protocols or adding new features to existing routing protocols. Fortunately, the two routing protocols in wide use in the Internet today, Open Shortest Path First (OSPF) and Intermediate System to Intermediate System (IS-IS), can be upgraded to include more sophisticated dynamic routing and TE features through reasonable changes and additions. Extensions to OSPF and IS-IS allow nodes to exchange information about network topology, resource availability, and policies. These extensions, although simple, have a profound impact on the way IP networks will evolve to provide new classes of guaranteed services for more advanced applications and offer support for the true convergence of (non-realtime) data and real-time multimedia traffic. This article presents several results that demonstrate the power of the recently proposed TE extensions to OSPF, currently known as OSPF-TE, and their impact on the routing process in IP networks. In particular, we focus on the fundamental role of OSPF-TE in enhancing a network’s ability to reserve and distribute bandwidth resources to traffic aggregates. We start by IEEE Communications Magazine • May 2004 0163-6804/04/$20.00 © 2004 IEEE 151 One operational scenario is to use OSPF-TE or ISIS-TE to compute best paths in a network then use MPLS in conjunction with a signaling protocol, such as RSVP-TE or CRLDP, to “pin down” trafficengineered paths, or tunnels, across the IP network. presenting a brief overview of the OSPF protocol, followed by a more detailed overview of OSPFTE. An example of OSPF-TE operation on a simple network is presented. We present the results of a performance study on OSPF-TE. We discuss the traffic overhead generated by OSPF-TE relative to that traffic generated by native OSPF, and the influence of link state advertisement (LSA) triggering techniques on QoS routing accuracy and traffic overhead under label switched path (LSP) reservations. The effects of MinLSInterval and MinLSArrival are also discussed. LINK-STATE BASED ROUTING OSPF was developed in 1988 by the OSPF working group of the Internet Engineering Task Force (IETF) and specified originally by Request for Comment (RFC) 1247 (OSPF version 1) [1], then by RFC 2328 (OSPF version 2) [2]. A new OSPF version for IPv6 is proposed in RFC 2740. Designed to run internal to an autonomous system (AS), OSPF is classified as an interior gateway protocol (IGP). The main function of OSPF is to distribute link state information across the network so that each network element is able to make routing decisions based on a consistent view of the network. OSPF is a link state routing protocol that often employs the well-known Dijkstra’s shortest path first (SPF) search algorithm [3]. As a link state routing protocol, OSPF allows each participating router to send LSAs, which contain information about the state of its interfaces, to all other routers within the same AS. Examples of such information are the routing metrics and the IP addresses of attached interfaces. Each router distributes its local state throughout the AS using a well-known technique called flooding. Initial exchange of LSAs allows routers to build an incremental view of the topology of the AS. This can be accomplished by having each router process and accumulate incoming LSAs into a special memory entity referred to as the link state database (LSDB). Eventually, each OSPF router accumulates enough link state information to end up with a database describing the topology of the entire AS. An AS can be divided into OSPF “areas” for scalability, and in normal steady-state operation all routers in the same OSPF area end up with identical LSDB contents, which contains detailed information on the topology of that area and a summary of other areas. Using the information in their respective LSDB, routers in the AS can run their own routing algorithms concurrently but independently. If a shortest path algorithm is deployed, the router constructs a tree of shortest paths to all other routers in the same AS with itself positioned at the root of the tree. The current OSPF protocol is not adequate for TE because the limited amount of information distributed by native OSPF is optimized for shortest path routing but insufficient for TE routing calculation. The shortest path routing scheme often used in OSPF, while sufficient to achieve connectivity, is not satisfactory from a TE point of view. For instance, in a typical OSPF network, congestion frequently occurs when [4]: • The shortest paths of multiple traffic streams converge on specific links or interfaces. • A given traffic stream is routed through a router interface that does not have sufficient link or processing resources to guarantee minimum bandwidth, maximum delay, and so on. Clearly, there is a need to enhance native OSPF to encompass some means to distribute additional state information required for TE so that traffic characteristics such as bandwidth availability are considered in the route selection decisions. This implies that each router should run a constraint-based routing algorithm to determine the best path rather than the usual shortest path with respect to network resource utilization and traffic performance. In constraint-based source routing, a router can compute a path subject to various imposed constraints on the attributes of the links and nodes the path traverses. The TE extensions to OSPF allow routers to exchange information about network topology, resource availability, and policies. This information can be used by the constraint-based routing algorithms to compute best paths subject to specified resource and policy constraints. Although the extensions are still new in the literature realm, we have already seen some live industry examples of these extended protocols. For instance, multiprotocol label switching (MPLS) is currently being developed as a means by which IP networks can provide connection-oriented services where QoS and TE can be enforced more effectively. Indeed, deployment of MPLS for traffic engineering applications has commenced in some service provider networks. One operational scenario is to use OSPF-TE or ISIS-TE to compute best paths in a network, then use MPLS in conjunction with a signaling protocol such as Resource Reservation Protocol with TE (RSVPTE) or Constraint-Based Routed Label Distribution Protocol (CRLDP) to “pin down” TE paths, or tunnels, across the IP network. OSPF-TE OVERVIEW As mentioned above, TE enhancements to current link state IGPs such as OSPF require the distribution of additional state information necessary for constraint-based routing. The additional information can be propagated in LSAs. Part of the additional topology state information includes link attributes such as reservable bandwidth and link resource class, an administratively specified property of the link. OSPF-TE, the TE extensions to OSPF, is described in detail in two IETF drafts, “Traffic Engineering Extensions to OSPF Version 2” [5] and “Traffic Engineering Extensions to OSPF Version 3” [6], as works in progress. In this article we base our discussion on OSPF-TE as described in [5]. It is worth pointing out that many of the extensions proposed in OSPF-TE are in response to RFC 2702, which is concerned with requirements for traffic engineering over MPLS [4], and are also commonly associated with MPLS traffic engineering. Built on native OSPF, OSPF-TE uses the opaque LSA of type 10 to disseminate TE information (see definitions below). This (TE) opaque LSA carries a special type of TLV triplet (see definitions below) called a link TLV. Link 152 IEEE Communications Magazine • May 2004 TLV encodes information about a specific link. The information includes the maximum bandwidth of the link, the maximum reservable bandwidth on the link (which may be different from the maximum bandwidth if links are allowed to be oversubscribed), the unreserved bandwidth for eight different priority levels, in addition to other information. The processing of the opaque LSA is the same as other LSA types; flooding is still used to distribute LSAs so that all the routers will have the same LSDB. In OSPF-TE, this database is called the TE database (TED) and contains the TE information of every link in the network. Using the entries in this database, the routers are able to compute TE paths (e.g., end-to-end MPLS paths with certain QoS capabilities). Definitions •The TLV construct is a method of encoding protocol messages in an extensible manner. Essentially, each parameter in a message is represented by three fields: type (T), length (L), and value (V). The type field is an implementation specific value that identifies the type of information carried in the packet. The length field specifies the length of the V field of the packet in bytes (or octets). The value field contains the actual value of the information type. •Opaque LSAs were defined in IETF’s RFC 2370 [7] as additional enhancements to OSPF. An opaque LSA consists of a standard LSA header followed by a 32-bit aligned applicationspecific information field that adds more attributes to OSPF LSAs. The additional information field may be used directly by OSPF or by other applications wishing to distribute information throughout the OSPF domain. Three opaque LSAs (types 9, 10, and 11) have been defined for OSPF. Only type 10 opaque LSA is used in the current OSPF-TE proposal. 0 8 LS age 1 16 Options Instance Advertising router LS sequence number 24 10 31 LS checksum TLV type TLV value... Length TLV length The value itself can be composed of multiple nested sub-tlvs. I Figure 1. Opaque TE-LSA format. to the headers of other LSAs as defined in OSPF v. 2, except for the following changes. The link state type is set to 10, signifying that this is an opaque LSA with area-flooding scope. Like any other opaque LSA, the link state ID portion of the native OSPF header is divided into two fields: an 8-bit opaque type field and a 24-bit opaque ID field. All TE-LSAs have the opaque type field set to 1, and the opaque ID is referred to as the instance in OSPF-TE. This instance field, along with the advertising router field, allows a router to differentiate between TELSAs without referring to the contents of the payload. Lastly, the second bit in the options field (unused in the original OSPF v. 2, but defined to be the O-bit in [7]) should be set to 1 by the advertising router, indicating that it is willing to receive and forward opaque LSAs. The link TLV describes a single link by including a set of sub-TLVs to describe finergranularity changes in the network topology. The current OSPF-TE proposal allows for only one link TLV to be carried in each LSA. The following sub-TLVs are defined: • Link type (1 octet) • Link ID (4 octets) • Local interface IP address (4 octets) • Remote interface IP address (4 octets) • Traffic engineering metric (4 octets) • Maximum bandwidth (4 octets) • Maximum reservable bandwidth (4 octets) • Unreserved bandwidth (32 octets) • Administrative group/resource class/color (4 octets) The details of the sub-TLVs are documented in [5]. Here, we briefly describe the four subTLVs used in our network simulation model and explain the role of each. Maximum bandwidth: This sub-TLV specifies the maximum bandwidth that can be used on the link or advertising interface from the router originating the LSA to one of its neighbors. This is the true link capacity. Maximum reservable bandwidth: This subTLV specifies the maximum bandwidth that may be reserved on a given link from the router originating the LSA to one of its neighbors. When link bandwidth oversubscription is allowed, the maximum reservable bandwidth may be greater than the maximum bandwidth. This sub-TLV is user-configurable, and the default value should be set to the maximum bandwidth. OSPF-TE PROTOCOL DESCRIPTION AND PACKET FORMAT The OSPF-TE makes use of the type 10 opaque LSA. For the purpose of TE, one new LSA is defined, the TE-LSA. The TE-LSA describes routers, point-to-point links, and connections to multi-access networks in a way similar to a router LSA. For multi-access links, the existing type 2 network LSA is sufficient, so no additional LSA is defined for this purpose. The packet format for the TE-LSA is shown in Fig. 1. As shown, the TE-LSA starts with the standard (or native) OSPF LSA header followed by the payload, which consists of one or more nested TLV triplets for extensibility. An LSA contains a single top-level TLV. Two top-level TLV types are defined: router address and link. The router address TLV specifies a stable IP address of the advertising router that is always reachable as long as there is any connectivity to the router. This IP address is typically implemented as the router loopback address or what is known as the router ID, and has the advantage that it would not become unusable if the particular router interface is down. The router address TLV must appear in exactly one TE-LSA originated by a router. The header format of a TE-LSA is identical IEEE Communications Magazine • May 2004 153 TE policy manager (operator) Path selection Resource signaling (RSVP-TE or CR-LDP) Reserve path Bandwidth manager TE database Flooding OSPF-TE Routing Path admission LSAs Packet forwarding engine I Figure 2. OSPF-TE’s role in the constrained routing and path selection process. router will also use Network LSAs, which are type 2 non-opaque LSAs, to transport information used for traffic engineering calculations. Upon receipt of a changed TE-LSA or a network LSA, the router updates or builds its TE database, which contains additional link attributes to the regular LSDB. This update does not necessitate shortest path or other route computations. The router decides on when to perform such calculation based on its own state and timing configuration. An OSPF-TE router can participate in routing within an OSPF area, update its TED, and report on the reservation state of links in that area. Using its TED, the router can run standard shortest path or constraint-based source routing to compute a path from the router to a destination node subject to constraints on the attributes of the links and nodes its path traverses. For example, the algorithm may search for a path from one of its interfaces to a given area border router that provides a minimum bandwidth of 10 Mb/s, or one that provides a delay guarantee for a given traffic class. However, constraint-based routing is outside the scope of this article and is the subject of a companion publication by the authors [8]. Unreserved bandwidth: This sub-TLV specifies the amount of link bandwidth not yet reserved at each of the available eight priority levels. The values correspond to the bandwidth that can be reserved with a setup priority of 0 through 7. The initial values are all set to the maximum reservable bandwidth. Afterwards, each value will be less than or equal to the maximum reservable bandwidth. Traffic engineering metric: This sub-TLV specifies a metric for the link based on TE considerations and may be different from the standard metric used by native OSPF. Figure 2 shows a typical system view of the routing and path selection component of an OSPF-TE IP router. As shown, OSPF-TE plays a key role in transferring dynamic link state information to build the TED. This database is used by the path selection block, which is essentially a constraint-based source routing algorithm, to determine the best (not necessarily shortest) available paths under the specified constraints. Other uses of the TED include monitoring the extended link attributes and building multiple topological or logical views of the network for global traffic engineering. The resource signaling block propagates messages to reserve the paths computed by the path selection block for admitted traffic aggregates. The bandwidth manager is essential for providing information on the amount and availability of bandwidth on the link. AN EXAMPLE OF OSPF-TE OPERATION To illustrate how OSPF-TE operates, consider the simple four-router network shown in Fig. 3. The routers on the left and right ends are edge routers connected to the external network by 1-Gb point-to-point links. The two routers in the middle bridge the edge routers with 100 Mb/s pointto-point links. Each of the links in the top path (through routers R1, R2, R4) has a link cost metric of 1, while each of the links in the bottom path (through routers R1, R3, R4) has a cost of 10. Assume there are two traffic flows, labeled A and B, each with a bandwidth requirement of 60 Mb/s, both entering the network through the left router (R1) and exiting through the right router (R4). It is obvious that the combined traffic load from the two flows has to be shared between the top and bottom paths in Fig. 3, since neither path is able to accommodate both flows at the same time. With native OSPF and basic packet switching, it is impossible to share the traffic load between the two paths, a shortcoming of the shortest path paradigm. The packets from both flows would be routed to one of the paths, in this case the top path, which has the lower link cost. As a result, the bandwidth requirements of both flows would not be satisfied, while the bottom path remains largely underutilized. Note that local flow scheduling would not solve the problem for both flows. For example, if flow A is given priority over flow B, all the packet loss will be confined to flow B, so at least one of the flows will suffer substantial packet loss. The solution to the above problem is to use a connection-oriented path switching scheme such as MPLS, in which signaling protocols such as RSVPTE or CR-LDP are used to set up a path for each flow, reserving resources along the path to ensure that the bandwidth requirements are satisfied. The THE MAIN ROUTING PROCESS It should be clear from the previous presentation that TE extensions to OSPF mainly provide extended link attributes, since what is proposed simply adds more attributes to the LSAs. In all other respects, OSPF-TE operates quite similar to native OSPF. An OSPF-TE router originates TE-LSAs (type 10 opaque LSAs) when either the LSA contents change or otherwise required by OSPF (e.g., to perform an LSA refresh). The 154 IEEE Communications Magazine • May 2004 R2 202.0.0.4 /s Mb 00 c = 1 1 ri t Me 202.0.0.5 M 101.0.0.1 202.0.0.1 1 Gb/s R1 10 0 etr Mb/ s ic = 1 202.0.0.8 202.0.0.9 R4 202.0.0.10 303.0.0.1 1 Gb/s 202.0.0.2 202.0.0.3 Me 10 tri 0M Flow A 60 Mb/s Flow B 60 Mb/s b c = /s 10 R3 0 M 10 10 ic = tr Me b/s 202.0.0.6 202.0.0.7 I Figure 3. An example of an OSPF-TE network. actual procedure of choosing the path is implementation-specific, and route determination can be performed completely at the ingress node or be distributed across the nodes along the path (i.e., the nodes along the path decide which node will be the next hop). We will next compare what would happen if OSPF or OSPF-TE were used with a signaling protocol, with the assumption that the complete route is computed by the ingress node (router R1 in Fig. 3). After the path is set for flow A, the available bandwidths on the links along the path change. Depending on the OSPF-TE implementation, the following could happen: • If link state updates were implemented to occur periodically, the routers along the path would distribute the newest link state attributes after a short period of time (a few seconds). • If the triggering of link state updates were based on the amount of change in unreserved bandwidth, an update may or may not occur. In our example, an update would probably take place immediately, since there is a major change in available bandwidth. Note that the triggering method for link state updates is not part of the OSPF-TE specification. In either case, the routers would create new TE-LSAs carrying the latest information and flood them across the area. Assume that a link state update has occurred as a result of the path reservation for flow A, and that flow B arrives after the update. The ingress router would know that the top path is congested. Without attempting to reserve the top path, the routing algorithm would look for a path with sufficient bandwidth. In our example, the bottom path would be chosen using information available for the constrained shortest path algorithm. No trial and error is involved in the procedure, and no bandwidth is wasted by the signaling protocol, although OSPF-TE induces some bandwidth overhead for the distribution of the additional link state attributes. It is possible that flow B may start before the link state update, resulting in an unsuccessful attempt to reserve the top path. This is more likely to happen if the triggering scheme was suboptimal, or there were other factors such as a very short interarrival time between flows A and B. NATIVE OSPF WITH RSVP-TE/CR-LDP With native OSPF, the only information distributed across the network is the presence of routers, the existence of links between them, and the metric costs of the links. When the first flow, say flow A, is initiated, the ingress router would choose the shortest path for the flow, which is the top path in the example of Fig. 3. The signaling protocol would then be used to reserve the resources for flow A (i.e., 60 Mb/s of link bandwidth) along the top path. When flow B starts, the ingress router again has to choose a path for it. However, the ingress router does not know that the top path is too narrow for B, and since the top path still has a lower cost it will be chosen by OSPF. In this situation, the signaling protocol would be unable to reserve the 60 Mb/s required for flow B, and would report to the ingress router a failure of path setup due to insufficient bandwidth. Depending on the implementation of the routing algorithm, the ingress router may finally realize that the bottom path is possibly a good choice and proceed to set up the path for flow B. Setting up the path requires trial and error, resulting in some bandwidth being wasted by the signaling protocol. In a more complex network this trial and error process can pose a problem as there are multiple possible parallel paths and the ingress router may attempt several congested but lower-cost paths before selecting a path with sufficient available bandwidth. OSPF-TE WITH RSVP-TE/CR-LDP With OSPF-TE, in addition to the information propagated in the native OSPF network, the available bandwidths of each link is also distributed across the area. When flow A starts, the ingress router chooses the top path in Fig. 3 with the lower link costs, as in native OSPF. PERFORMANCE OF LINK STATE UPDATE MECHANISMS IN OSPF-TE The new features and usages of OSPF-TE introduce new capabilities to the routing processes and at the same time pose new challenges to system IEEE Communications Magazine • May 2004 155 It should be noticed that the number of links per router (i.e., the node degree) is the main factor affecting the amount of overhead generated by OSPF-TE. This is because most of the protocol traffic results from the router advertising LSAs for each of its links. design. Among the challenges is the extra TE link state information propagated, which results in higher amounts of protocol traffic than in native OSPF. Another challenge is that many of the new link state attributes, such as the available link bandwiddh, are dynamic in nature and require timely LSA updates. Indeed, the TE-based triggering mechanisms for LSA update are very crucial to the successful deployment of OSPF-TE in future networks. In this section we report several key results on the effectiveness and cost performance trade-offs of various LSA triggering mechanisms. The appendix provides the details of the simulator we have employed in our experiments. The results are broadly divided into two main parts. In the first part OSPF-TE is compared to native OSPF mainly in terms of the protocol packets generated by both schemes during the period required to synchronize the LSDBs (or TEDs in OSPF-TE) in all the routers across the OSPF network. In the second part we compare several existing and new mechanisms for triggering LSA updates in an OSPF-TE network that allows path reservations using an additional signaling protocol. All the simulation runs were repeated eight times and have a confidence interval above 95 percent. OSPF-TE PROTOCOL OVERHEAD In this section we compare the protocol traffic overhead of OSPF-TE vs. native OSPF. Since OSPF-TE supports the advertisement of more link attributes, one can expect it to cause more LSA flooding traffic. We have simulated four networks with different topologies that capture variations in connectivity, network size, and number of areas. The following networks are used in our experiments: • A very small network consisting of two nodes only • A heavily connected network with 16 routers • A moderately connected network with a tree topology with 14 routers • An OSPF-TE network with 12 routers split into two areas In all cases, the routers are connected by synchronous optical network (SONET) OC-1 pointto-point links. In the experiments we set up the topologies and run the simulations without any real-time link state changes (i.e., no link/node failures and no path reservations). The routers establish adjacencies and advertise all the necessary LSAs once. Thereafter, the protocol traffic is generated via the exchange of Hello messages. Note that the protocol traffic for an OSPF-TE network is the sum of native OSPF traffic and the additional traffic generated by TE-LSAs. The recorded statistics from the simulations represent all OSPF (or OSPF-TE) related traffic, including LSA flooding as well as Hello message exchanges. The topologies and respective results are shown in Fig. 4. For the two-router network of Fig. 4a, the protocol traffic generated by OSPFTE is shown to be about 79 percent higher than native OSPF. For the moderately connected tree network of Fig. 4b, OSPF-TE generates 156 percent more traffic. In the heavily connected 16router network of Fig. 4c, OSPF-TE generates 255 percent more traffic. It should be noticed that the number of links per router (i.e., the node degree) is the main factor affecting the amount of overhead generated by OSPF-TE. This is because most of the protocol traffic results from the router advertising LSAs for each of its links. In OSPF-TE, these advertisements are performed per link; thus, more links would generate more traffic. In native OSPF, the increased number of links does not dramatically increase traffic since link information is advertised per router. Figures 4a–c provide quantitative results confirming this observation. It is important to note that the above results are relevant for point-to-point links only. If the routers were connected by Ethernet links, a native OSPF router would see the link itself as a reachable network, and generate a network LSA for each link. In this case the native OSPF traffic would increase for each additional link, similar to OSPF-TE. To control the protocol traffic load, an OSPFTE network can be partitioned into multiple areas. Figure 4d shows that by partitioning the network shown into two areas, the OSPF-TE protocol traffic can be reduced by approximately 60 percent. This, of course, comes at a price since OSPF-TE flooding traffic is exclusively intra-area; TE information is not propagated across areas within the same network. Therefore, network-wide TE-based routing is not possible without an external means to propagate TE information across area borders. Such external means are outside the scope of current OSPFTE specifications. OSPF-TE PERFORMANCE UNDER TE PATH RESERVATION While routing algorithms for path selection are out of the scope of OSPF-TE, it is the responsibility of OSPF-TE to distribute up-to-date dynamically changing TE link state information throughout the network to ensure accurate routing. This information can be used by other path setup mechanisms, such as RSVP-TE and MPLS, to reserve and establish TE tunnels or label switched paths (LSPs) across a network based on TE information supplied OSPF-TE [4]. To maintain specificity, we will use the term LSP to refer to a reserved path. However, the same treatment applies to other path reservation schemes. As will be shown, the performance of OSPFTE depends rather strongly on the triggering scheme for LSA updates. The triggering scheme determines when a router should originate a TELSA to advertise a change in link state attribute. A good triggering scheme should maximize routing accuracy while minimizing the LSA flooding traffic due to OSPF-TE. We have performed extensive simulations to explore how different triggering schemes affect the performance of OSPF-TE in terms of routing accuracy and flooding traffic. In particular, we study the effect of choosing specific values of the MinLSInterval and MinLSArrival fixed timers, intrinsic to OSPF and OSPF-TE, on routing performance. Mechanisms for Reducing TE-LSA Flooding — Generating a new TE-LSA update after every bandwidth reservation maximizes the routing accuracy, but also increases the network load dramatically, especially if the frequency of estab- 156 IEEE Communications Magazine • May 2004 Two nodes 250 Traffic sent (b/s) 200 150 100 50 0 144 216 288 360 (a) OSPF OSPF-TE 360 (b) OSPF OSPF-TE 15,000 10,000 5000 0 144 216 288 360 (c) 72 0 72 0 OSPF OSPF-TE Time (s) Tree topology 10,000 Traffic sent (b/s) 8000 6000 4000 2000 0 144 216 288 72 0 Time (s) Heavily connected (16 nodes) 20,000 Traffic sent (b/s) Time (s) One vs two areas 7000 Traffic sent (b/s) 6000 5000 4000 3000 2000 1000 0 144 216 288 360 72 0 OSPF one area OSPF-TE one area OSPF two areas OSPF-TE two areas Area 2 Area 0 Area 1 Time (s) (d) I Figure 4. OSPF load vs. OSPF-TE load in different networks: a) two-router (node) network; b) tree-type (sparse) network; c) mesh (dense) network; d) multi-area network. IEEE Communications Magazine • May 2004 157 Generating a new TE-LSA update after every bandwidth reservation maximizes the routing accuracy, but it also increases the network load dramatically, especially if the frequency of establishing and tearing down reservations is high. lishing and tearing down reservations is high. By observing the TE-LSA updates sent across the network, it is easy to observe that many such updates are unnecessary since they do not add critical information to the recipient routers. Given this fact, one of our experimental goals was to decrease the network load by limiting the number of TE-LSA updates flooded to the area through the application of such techniques as periodic updates or thresholding mechanisms, defined as follows: Periodic updates: In this technique the router will generate a new LSA update periodically according to a predetermined period. The newly generated LSA will be flooded regardless of whether or not it has new information. Delta thresholding: A new LSA will be generated and flooded whenever the difference between the current link available bandwidth and the last flooded bandwidth exceeds a certain constant threshold. Relative thresholding: A new LSA will be generated and flooded whenever the difference between the current link available bandwidth and the last flooded bandwidth exceeds a certain percentage (Threshold_Value) of the last flooded bandwidth, according to the following formula: Last Flooded Bandwidth − Current Unreserved Bandwidth Relative = Theshold Last Flooded Bandwidth IF Relative Threshold ≥ Threshold_Value THEN Flood Native OSPF (v. 2) also specifies two fixed timers, MinLSInterval and MinLSArrival, that can be used to limit the amount of flooding traffic in an OSPF or OSPF-TE network. Timer MinLSInterval is applied at the router originating the LSAs and is defined as the time interval within which the router can originate at most one instance of any particular LSA. This timer is set to 5 s in the current OSPF specification. The MinLSArrival is set to 1 s and is referenced by all other receiving routers in the network. Essentially, a receiving router will not accept a newer LSA if a previous copy of the LSA was received less than MinLSArrival seconds earlier. In the current Internet draft for OSPF-TE, it is recommended for originating routers to conform to the MinLSInterval requirement. These two timers, in addition to the periodic and threshold-based traffic-limiting mechanisms, can be applied to all types of LSAs, not only the TE-LSAs. Of course, reducing the LSA flooding often results in loss of synchronization between TEDs of different routers. This loss of synchronization introduces errors in the routing decisions made by the ingress routers. In order to measure the effect of this problem, the following statistics were collected. Over reservation: This phenomenon occurs whenever a router makes the “bad” decision that a certain path is feasible (i.e., has sufficient bandwidth) according to stale information in its TED, when in fact the particular path does not currently have sufficient bandwidth. In our simulator, a special reservation error counter is incre- mented by 1 each time such a bad decision is made by the router. Under reservation: This occurs whenever a router makes the bad decision that a certain path is not feasible (i.e., does not have enough bandwidth) according to stale information in its TED when in fact the particular path currently has sufficient bandwidth. In our simulator, the special reservation error counter is incremented by 1 each time such a bad decision is made by the router. Path reservation decisions that cause under or over reservation are considered to be bad decisions. Any path reservation decision other than the two listed above is considered to be a “good” decision. The effectiveness of the protocol traffic-limiting techniques described above will be evaluated in terms of the ratio of bad decisions they cause an ingress router to make relative to the total number of path reservations decisions made by the same router. Performance with MinLSInterval and MinLSArrival Disabled — Figure 5 shows the simple topology we chose to test OSPF-TE performance under path reservations. Routers R1 and R3 generate LSP reservation requests, both to reach router R6. Router R1 has two available paths to reach R6, one through R2 and the other through R4 (the route through R2 being the least cost path). Router R3 has only one available path, through R2 then to R6. For simplicity, we have excluded route R3-R2-R1R4-R5-R6 from consideration. Main assumptions: We assume that both R1 and R3 generate LSP requests with exponentially distributed request inter-arrival time and LSP hold duration with a mean of 15 s and 180 s, respectively, and a uniformly distributed bandwidth requests ranging from 64 kb to 5 Mb [9]. It is assumed that all the routers are OSPF-TE-capable and the links are point-to-point SONET OC-1 links. The arrivals statistics were chosen carefully to guarantee that the link between R2 and R6 becomes congested at some point. Whenever the link between R2 and R6 becomes congested and unable to guarantee the requested bandwidth, R1 can reroute the path to R1-R4-R5-R6, while R3 will reject the LSP request to R6. Figure 6 presents simulation results when both MinLSInterval and MinLSArrival are disabled (i.e., both set to 0 s). This has the effect of immediately flooding LSAs after the update period has elapsed or the threshold level has been crossed. Figure 6a1 shows the results obtained when using periodic updates. In Fig. 6a1, the horizontal axis represents the LSA update period, and the vertical axis is the ratio of bad decisions made by the ingress routers (R1 and R3). In Fig. 6a2, the vertical axis indicates the average OSPF-TE traffic sent by R2. Figure 6a1 shows how the ratio of bad decisions increases as the TE-LSA update period is increased from 5 to 20 s, while Fig. 6a2 shows how the average traffic flooded by R2 decreases as the update period is increased. This trade-off between the ratio of bad decisions (or routing accuracy) and the OSPF-TE load will also be observed with the other mechanisms. In Figures 6b1 and 6b2, the horizontal axis represents the number of thresholds that divide 158 IEEE Communications Magazine • May 2004 the link bandwidth. For example, with five thresholds a 50 Mb/s link is divided into five bandwidth bands of 10 Mb/s each, with the threshold barriers located at 0, 10 Mb/s, 20 Mb/s, 30 Mb/s, and 40 Mb/s. With 10 thresholds the link bandwidth will be divided into 10 bands of 5 Mb/s each, and so on. Each point on the horizontal axis represents a repetition of the experiment with new number of thresholds applied to the link bandwidth. When a new LSP reservation causes the link bandwidth to exceed a threshold value, a TE-LSA will be originated by the router receiving the request. The same applies when a reservation is torn down. These results show that increasing the number of thresholds improves routing accuracy by reducing the ratio of bad decisions made by the ingress routers for LSP reservations, but at the expense of increasing LSA traffic in the network. Finally, the horizontal axis in Figs. 6c1 and 6c2 represents different settings of the relative threshold, ranging from 0 to 1. In experiments that use small values of the relative threshold (e.g. 0, 0.1, and 0.2), TE-LSAs tend to be generated more often following small changes in link bandwidth, resulting in a small ratio of bad reservation decisions. Table 1 provides a sample comparison of the three LSA update mechanisms explained above to meet the condition that the ratio of bad decisions made by the ingress routers for LSP reservations must be kept under 5 percent. The important observation for Table 1 is that relative thresholding provides the same level of routing accuracy decisions as the other two mechanisms, but with substantially less LSA traffic. The results of Table 1 are derived from the results of Fig. 6. Performance with MinLSInterval and MinLSArrival Enabled — The results of the previous section clearly demonstrate the fundamental trade-off between routing accuracy and the amount of LSA flooding traffic when MinLSInterval and MinLSArrival are disabled,. In the following, we demonstrate how enabling MinLSInterval and MinLSArrival will impact the routing performance of OSPF-TE. In our implementation, when the MinLSInterval rule is violated at an LSA-originating router, origination of all new instances of the LSA will be deferred until MinLSInterval (i.e., 5) seconds from the last legal LSA origination. The concerned router then originates a single LSA containing the most up-to-date information. For example, suppose the unreserved bandwidth on a link was 10 Mb/s at time 100 s; there were two more successful LSP reservations at times 101 s and 102 s, each with 1 Mb/s bandwidth requirement. In this case, the LSA issued R3 LSP requests R3 to R6 R1 LSP requests R1 to R6 R4 R5 R2 R6 I Figure 5. Network topology under test. at time 100 s will advertise an unreserved bandwidth of 10 Mb/s on the link, but the effect of the two new successful reservations will be suppressed until 105 s, at which time the ingress router will originate a single LSA advertising 8 Mb/s of unreserved bandwidth. For an OSPF-TE network with persistent LSPs (e.g., for voice over IP), where the network state changes are less frequent, MinLSInterval and MinLSArrival will have less significant effects on routing performance since it is less likely for both timers to be violated by the LSA generation process. However, in the case of a network with more dynamic LSPs with shorter hold times, network states (particularly the unreserved bandwidth on links) may change more than once within a 5-s (MinLSInterval) period. It is obvious that enabling the MinLSInterval timer would negatively affect the performance of the thresholding schemes by delaying a significant portion of LSA generation. Periodic thresholding, on the other hand, would be largely unaffected as long as the period is longer than MinLSInterval. Figure 7 shows the simulation results from applying relative thresholding. As in previous simulations, the network parameters were chosen so that the OC-1 link between R2 and R6 has an average load equal to 60.0768 Mb/s, slightly overloading the link. Simulations were performed for two cases: • Case I: Ingress routers R1 and R3 receive LSP requests with mean interarrival time of 15 s and a mean LSP duration (or hold time) equal to 180 s. The request bandwidth is uniformly distributed from 64 kb/s to 5 Mb/s. This case is depicted in Fig. 7a. • Case II: Ingress routers R1 and R3 receive LSP requests with mean interarrival time of 2 s and a mean LSP duration of 24 s. The request bandwidth is uniformly distributed from 64 kb/s to 5 Mb/s. This case is depicted in Fig. 7b. Periodic update Condition met at Corresponding mean traffic sent by router R2 Period = 5 s 6000 b/s Delta threshold Number of thresholds = 20 700 b/s Relative threshold Threshold = 0.7 250 b/s I Table 1. A comparison of different LSA update mechanisms to satisfy the condition: keep the ratio of bad decisions for LSP reservations below 0.05 (i.e., below 5 percent). IEEE Communications Magazine • May 2004 159 Bad decisions Mean traffic sent (b/s) Percentage of bad decisions 0.2 0.15 0.1 0.05 0 5 10 15 20 25 30 Period (s) (a1) Bad decisions 0.2 0.15 0.1 0.05 0 100 5 20 40 60 80 Mean traffic sent (b/s) Percentage of bad decisions 0.25 1500 1000 500 0 8000 6000 4000 2000 0 5 10 Mean traffic sent 15 20 25 30 Period (s) (a2) Mean traffic sent Number of thresholds (s) (b1) Bad decisions 0.2 0.15 0.1 0.05 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Threshold (percentage) (c1) Mean traffic sent (b/s) Percemtage of bad decisions 0.25 1500 1000 500 0 0 0.1 Number of thresholds (s) (b2) Mean traffic sent 100 1 20 40 60 10 30 50 70 80 5 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Threshold (percentage) (c2) I Figure 6. Performance of OSPF-TE when MinLSInterval and MinLSArrival are disabled: a1): percentage of bad decisions made by ingress routers using periodic updates; a2): OSPF-TE load under LSP reservations using periodic updates; b1): percentage of bad decisions made by ingress routers using delta thresholding; b2): OSPF-TE load under LSP reservations using delta thresholding; c1): percentage of bad decisions made by ingress routers using relative thresholding; c2): OSPF-TE load under LSP reservations using relative thresholding Table 2 shows the reduction in the amount of flooding traffic generated by OSPF-TE based on case I. Enabling the two rate-limiting timers significantly affects the amount of flooding traffic, reducing it by as much as 18.4 percent when the threshold is set to zero, but its effect becomes gradually less significant for higher threshold values. It should be emphasized that the particular threshold values shown in Table 2 should not be regarded as a general rule of thumb. One can expect that as the interarrival time of LSP requests decreases, the effect of MinLSInterval and MinLSArrival become more profound. As for routing accuracy, it is still true that when the threshold is high, say in the range 0.9–1, the accuracy would deteriorate. However, with MinLSInterval and MinLSArrival enabled, the ratio of bad decisions no longer improves steadily as the relative threshold is decreased. In fact, closer scrutiny of Fig. 7b reveals that, in both cases, there is a mid-level threshold that would provide maximum accuracy with minimum flooding traffic. In other words, there is a region on the low threshold range where increasing the threshold would both decrease flooding traffic and improve routing accuracy. This phenomenon is caused by two factors. First, it is obvious that with a very low threshold, more LSA updates are attempted, and thus MinLSInterval is frequently violated, and a large portion of link state updates are delayed. In the extreme case, the MinLSInterval would be constantly violated, and the system would degenerate into an equivalence of periodic thresholding with period equal to MinLSInterval. Increasing the threshold value would naturally decrease the number of LSA updates, and more important link state changes (e.g., more significant changes of unreserved bandwidth) would be advertised in a more timely manner. The less obvious reason of the two is the cascading effect of MinLSInterval delays. As mentioned before, when MinLSInterval is violated, a new LSA update would be originated after the time period had elapsed. The problem is that this LSA regeneration would cause another vulnerable period of MinLSInterval seconds, during which any LSA updates would be suppressed and regenerated later. Consider the extreme case, where we have an LSA regeneration at 100 s and incoming flows with interarrival time of 160 IEEE Communications Magazine • May 2004 90 Bad decisions % of bad decisions 0.2 0.15 0.1 0.05 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Threshold (%) (a) % of bad decisions 0.25 0.25 0.2 0.15 0.1 0.05 0 0 0.1 0.2 0.3 Bad decisions 0.4 0.5 0.6 0.7 0.8 0.9 1 Threshold (%) (b) I Figure 7. a): Percentage of bad decisions made by ingress routers; using relative thresholding, with reservation requests mean interarrival time = 15 s; b): percentage of bad decisions made by ingress routers; using relative thresholding, with reservation requests mean interarrival time = 2 s. MinLSInterval seconds, starting at 101 s. Normally, all LSA updates due to these incoming flows would be advertised immediately without delay. However, due to the first regeneration at 100 s, the update at 101 s would be delayed to 105 s due to the first regeneration, and the update at 106 s would be delayed to 110 s, and so on. One can imagine that if the threshold were set to a higher level, some of the smaller flows would not trigger an update, and thus break the system out of the cascading effect. Case II of the simulation studies the effect of shorter interarrival time between successive LSP requests. As expected, the more frequent link state changes in case II decrease the accuracy of routing (i.e., increase the ratio of bad decisions). Note that while 0.3 is the relative threshold value that results in the most accurate routing decisions for case I (Fig. 7a), the corresponding value for case II is 0.8 (Fig. 7b). This indicates that in certain cases, advertising a few significant changes to link bandwidth immediately is more beneficial than advertising a larger number of smaller updates with more delays (caused by MinLSInterval). Another useful observation is that for backbone routers, where the links serve a large number of LSPs, using a larger relative threshold may improve both the flooding traffic and routing accuracy. Figure 8 shows the results of applying simulation cases I and II using Delta thresholding. Using these results, Table 3 shows the effect of MinLSInterval and MinLSArrival on the flood- ing traffic sent by router R2. Again the rate-limiting timers reduce the amount of traffic significantly when the number of thresholds is large (or the threshold bandwidth is small). In this particular case, LSA flooding traffic can be reduced by as much as 18.3 percent. Of course, the degree of protocol traffic reduction decreases as the threshold size increases. As for relative thresholding, MinLSInterval and MinLSArrvial degrade the accuracy of delta thresholding, especially when the threshold range is very small. For case I with MinLSInterval and MinLSArrival enabled, the benefit of decreasing the threshold ceases when the threshold value is around 1/40 of the OC-1 link capacity (~1.3 Mb/s). Further decrease in the threshold size merely increases the amount of flooding traffic. For case II, the accuracy would not be significantly improved for threshold sizes below 1/10 of the OC-1 link capacity (~5.2Mb/s). CONCLUSIONS The traffic extensions to OSPF promise to provide more robust and powerful routing capabilities. The OSPF-TE proposal provides a practical enhancement to current OSPF since it involves only adding a new type 10 opaque LSA to the OSPF protocol, in addition to the required changes to the OSPF state machine. Although OSPF-TE has not been widely deployed yet, some protocol performance and % ofbad decisions Bad decisions 0.25 0.2 0.15 0.1 0.05 0 20 40 60 80 100 5 % of bad decisions 0.21 0.2 0.19 0.18 0.17 0.16 20 Bad decisions 40 60 80 Number of thresholds (a) Number of thresholds (b) I Figure 8. a): Percentage of bad decisions made by ingress routers; using delta thresholding, with reservation requests mean interarrival time = 15 s; b): percentage of bad decisions made by ingress routers; using delta thresholding, with reservation requests mean interarrival time = 2 s. IEEE Communications Magazine • May 2004 100 5 161 Relative threshold With MinLSx disabled With MinLSx enabled Changes (%) 0 1425 1163 –18.4 0.2 601 521 –13.4 0.4 386 355 –8.0 0.6 284 271 –4.5 0.8 216 212 –1.6 1.0 118 118 –0.3 REFERENCES [1] J. Moy, “OSPF Version 2,” RFC1247, July 1991, http:// www.ietf.org/rfc/rfc1247.txt. [2] J. Moy, “OSPF Version 2,” RFC 2328, Apr. 1998, http:// www.ietf.org/rfc/rfc2328.txt [3] E. W. Dijkstra, “A Note on Two Problems in Connection with Graphs,” Numerische Mathematik, vol. 1, 1959, pp. 269–71. [4] D. Awduche et al., “Requirements for Traffic Engineering over MPLS,” IETF RFC 2702, Sept. 1999. [5] D. Katz, K. Kompella, and D. Yeung, IETF Draft: “Traffic Engineering Extensions to OSPF Version 2,” draft-katzyeung-ospf-traffic-10.txt, work in progress. [6] K. Ishiguro and T. Takada, “Traffic Engineering Extensions to OSPF version 3,” IETF draft, draft-ietf-ospfospfv3-traffic-00.txt, work in progress. [7] R. Coltun, “The OSPF Opaque LSA Option,” IETF RFC 2370, July 1998. [8] H. M. Alnuweiri, J. Khan, and L.-Y. K. Wong, “Performance of OSPF-TE with Constraint-Based Routing,” 2003. [9] G. Apostolopoulos et al., “Quality of Service Based Routing: A Performance Perspective,” SIGCOMM ’98, 1998. [10] A.Basu and J. G. Riecke, “Stability Issues in OSPF Routing,” SIGCOMM ’01, Aug. 2001. [11] P. Narvaez, K.-Y. Siu, and H.-Y. Tzeng, “Traffic Engineering with Traditional Routing Protocols,” IEEE/ACM Trans. Net., vol. 9, no. 6, Dec. 2001 pp. 706–18. [12] OPNET Technologies Inc., OPNET Manuals, 2000. I Table 2. Effects of MinLSInterval and MinLSArrival on the average OSPFTE traffic (bits per second) sent by router R2 for relative thresholding. Mean path request interarrival time = 15 s/node, mean duration = 180 s. Threshold as a fraction 0 of OC-1 link capacity With MinLSx disabled With MinLSx enabled Changes (%) 1406 1149 –18.3 1/100 1283 1081 –15.8 1/80 1244 1055 1/60 1190 1021 1/40 1069 937 –12.4 1/20 1 702 669 –4.8 117 118 +0.3 –15.2 –14.3 I Table 3. Effects of MinLSInterval and MinLSArrival on the average OSPFTE traffic (bits persecond) sent by router R2, for delta thresholding. Mean path request interarrival time = 15 s/node, mean duration = 180 s. BIOGRAPHIES stability studies have been published recently to compare it to existing native OSPF (v. 2) [10]. This article serves to explicate the performance trade-offs between routing accuracy and protocol traffic overhead in OSPF-TE resulting from applying different TE-LSA update mechanisms. It is observed that enabling the ratelimiting timers, MinLSInterval and MinLSArrival, significantly alters the basic trade-offs between protocol traffic and routing accuracy in OSPF-TE. Finally, it is worth mentioning that a certain level of traffic engineering can be realized with native OSPF with the help of a centralized entity that can configure OSPF weights based on network topology information and traffic demand estimates, as explained in [11]. Such methods present an alternative to the use of OSPF-TE, but nevertheless require modifications and additions to the OSPF network. HUSSEIN ALNUWEIRI ((hussein@ece.ubc.ca) is an associate professor in the Department of Electrical and Computer Engineering at the University of British Columbia. His research interests cover all aspects of routing, traffic engineering, and QoS mechanisms in packet networks. He is also working on developing algorithms and protocols for future wireless networks, switching and routing in optical networks, and realtime multimedia communications. He has represented the University of British Columbia at the ATM Forum, and has served as a delegate of the Standards Council of Canada for the MPEG-4 (ISO/IEC JTC 1/SC 29/WG 11) committee. He obtained his Ph.D. in computer engineering from the University of Southern California, Los Angeles, in 1989. He holds two U.S. patents and one international patent. LAI-YAT KELVIN WONG is completing his Master’s degree in the Department of Electrical and Computer Engineering at the University of British Columbia. His research interests include traffic engineering and routing protocols for the Internet. T ARIQ A L -K HASIB is completing his Master’s degree in the Department of Electrical and Computer Engineering at the University of British Columbia. His research interests include routing and scheduling algorithms for packet data networks. APPENDIX: OSPF-TE SIMULATOR To simulate the new protocol, we have implemented the changes necessary to support OSPF-TE in the OPNET network simulator environment [12]. The core of our simulator consists of the OSPF-TE module based on the OSPFv2 module, which is built into OPNET. The OSPFv2 module implements the functionalities of native OSPFv2, and is used as a component within the OPNET simulation models (or node models) of routers, workstations, and other network items. For the experiments, the process models and the associated external source code of the OSPFv2 module were modified and expanded to support basic OSPF-TE functionalities such as originating and processing TE-LSAs. Our OSPF-TE module replaces the OSPFv2 module in OPNET. A network node with an OSPF-TE module can do the following: • Originate, receive, and flood a TE-LSA advertisement • Create and destroy a TE-LSA • Insert and extract a TE-LSA to and from an OSPF message • Install and remove a TE-LSA in the link state database A user can choose a configuration that uses only a portion of the TLVs/sub-TLVs defined in the OSPF-TE Internet draft [5]. 162 IEEE Communications Magazine • May 2004